Automatic Methods for Hiding Latency in Parallel and Distributed Computation

Automatic Methods for Hiding Latency in Parallel and Distributed Computation

In this paper we describe methods for mitigating the degradation in performance caused by high latencies in parallel and distributed networks. For example, given any “dataflow” type of algorithm that runs in T steps on an n-node ring with unit link delays, we show how to run the algorithm in O(T ) steps on any n-node bounded-degree connected network with average link delay O(1). This is a signi...

Hiding Communication Latency in Data Parallel Applications

Interprocessor communication times can be a significant fraction of the overall execution time required for data parallel applications. Large communication to computation ratios of the tasks performed by these applications results in suboptimal performance when executed on data parallel architectures. We present an alternate architectural framework, referred to as concurrently communicating SIM...

Automatic Data and Computation Decomposition on Distributed Memory Parallel Computers1

On shared memory parallel computers (SMPCs) it is natural to focus on decomposing the computation (mainly by distributing the iterations of the nested Do-Loops). In contrast, on distributed memory parallel computers (DMPCs) the decomposition of computation and the distribution of data must both be handled—in order to balance the computation load and to minimize the migration of data. We propose...

Parallel computation framework for optimizing trailer routes in bulk transportation

We consider a rich tanker trailer routing problem with stochastic transit times for chemicals and liquid bulk orders. A typical route of the tanker trailer comprises of sourcing a cleaned and prepped trailer from a pre-wash location, pickup and delivery of chemical orders, cleaning the tanker trailer at a post-wash location after order delivery and prepping for the next order. Unlike traditiona...

Partial Solutions Manual Parallel and Distributed Computation: Numerical Methods